[agent-kb] Fveritas: the problem factory — a harness that generates, gates, and solves its own research problems - #280
Open
Fveritas wants to merge 25 commits into
Open
[agent-kb] Fveritas: the problem factory — a harness that generates, gates, and solves its own research problems#280Fveritas wants to merge 25 commits into
Fveritas wants to merge 25 commits into
Conversation
added 25 commits
July 27, 2026 13:55
…ap (QuantumBFS#112), held-out test set calibrated
…ief, figures, data
…gistration, and missing-card rule
- build_run.py materializes the gitignored challenge-run folder (tracks/agent-kb/results/.../run.json) so /challenge-report can find the sawtooth-erosion run; verified build_report.py + render_report.py render cleanly and the skill's scan picks up 2 completed figures - solution README: mentor quickstart table (5 prompts, expected output each), requirements.txt, harness-contribution list for PR element 1 - track README: pin the chosen target (issue QuantumBFS#133, anchor QuantumBFS#112) as the file itself instructs - verified end-to-end in a fresh venv: calibration CALIBRATED, demo 1 survivor/1 deferred/3 dead, sawtooth anchors green
…4 sequential-loop repositioning - pf/budget.py + pf/round2.py + run_learning_loop.py: round-2 fleet licensed by round-1 heuristics; hop budget waste 29% -> 0%, zero deaths, deferred card relaunched bigger with next-step recommendation - tests/test_learning_loop.py anchors (fleet licensing, fingerprint non-collision, static fire, round-1 waste anchor 18/63) - docs/design/problem-generation.md: v3 methodology (badness filter + decomposition granularity) + sequential decision loop addendum - docs/discussion/: Day-4 brainstorm log (repositioning rationale) - README: 'The loop closes' section + quickstart note; log.md Day 3/4
…QuantumBFS#133 capability checklist
…ors), deferred verdict, QMC routing - cards/round3/tfim-ratio-sqrt5-001.yaml: frozen gate (|R-sqrt5|/sigma_R, kill 2.0) - pf/tfim2d.py: chain/square/triangular/honeycomb PBC clusters, even-parity ED - tests/test_tfim2d.py: dimer exact, classical limit, strong-field bound, parity, Jordan-Wigner cross-check (1e-10), Binder limits — TDD, written first - round 1 died no_solver (registry gate), builder added, round 2 deferred (0.73): R = 2.186 +/- 0.068 vs sqrt5; sigma_R 5700x too large -> routed to sign-free QMC - brief + data + figure + 2 heuristics entries + telemetry both rounds - INTERFACE.md: tfim_2d registered
… anchors (provenance-tagged)
…clean-room reproduction verified - ARGUMENT.md: single human-readable doc arguing correctness (Bethe/XDiag anchors, blind calibration 5/5+3/3, frozen gates) and usefulness (teeth in both directions, learning loop 29%->0% waste, held-out generalization) - README: link ARGUMENT.md from the top for reviewers - reproduction verified in a fresh clone + fresh venv: all 5 README prompts green, including the optional learning loop
results/ stays out of git per submission rules, so copy the self-contained report.html (figures embedded as data URIs) + report.json + run.json into solutions/problem-factory/report/ and link it from the README top line.
Author
|
Reproducible prompt (challenge-report element) — from a fresh clone, each numbered line is one short prompt to the agent, or run the command directly. Full version with expected outputs: python3 -m pip install -r tracks/agent-kb/solutions/problem-factory/requirements.txt
cd tracks/agent-kb/solutions/problem-factory
Optional: close the learning loop — Verified on this branch before opening the PR: all |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Team
Challenge
tracks/agent-kb— the issue'sMethodfield is "Other"; agent-kb is the track on agents that do research and the knowledge bases that ground them.What's in this PR
Solution —
tracks/agent-kb/solutions/problem-factory/(one command:python3 run_demo.py):run_demo.py— first flight: 5 cards → 1 survivor, 1 deferred, 3 dead, three death causes caught by three different mechanisms (duplicate fingerprint, setup error via Bethe oracle, no signal)run_calibration.py— calibration gate re-derives the [challenge]: Close the certified energy bracket for the kagome Heisenberg antiferromagnet #124–[challenge]: Tighter provable Trotter error bounds for a concrete 2D Hamiltonian #128 quality class blind: 5/5 curated accepted, 3/3 controls rejected → CALIBRATEDrun_learning_loop.py— round-2 fleet licensed by round-1 heuristics: budget waste 29% → 0%, zero deaths; the loop closesrun_sawtooth.py/run_sqrt5.py— held-out generalization: issue [challenge]: How do exact localized-magnon states die? Erosion maps for flat-band frustrated magnets #112 solved at reconnaissance scale (closed-form anchors to 1e-8–1e-10, XDiag cross-checked); issue [challenge]: Is the ratio of transverse-field Ising critical points on the triangular and honeycomb lattices exactly √5? #148 TFIM-2D flight → deferred verdict with QMC routingheuristics/— the accumulating heuristics library (an issue-named deliverable), with growth metricslog.md,docs/design/, and reproducibility anchors undertests/(all green, deterministic — a rerun reproducesresults/byte-identically)Harness improvements —
.knowledge/models/cards for sawtooth-chain and transverse-field-ising, a provenance-tagged sawtooth localized-magnon solvable oracle, thequantum-modelmissing-card rule inAGENTS.md, and the agent-kb track README.Generator-side collaboration repo (teammate): https://github.com/Fveritas/Assembly